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Summary 

Work-based assessments (WBAs) were only recently 
introduced into medicine. However, since their introduc- 
tion, they have rapidly grown in popularity. WBAs are now 
a routine part of medical training. As WBAs are being 
implemented, the practical difficulties with their use have 
come to light. A major problem is failure of the trainees, 
trainers and training programmes to adequately engage 
with them. In this review, the reasons for this and how 
these can be overcome are discussed. 
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Introduction 

Medical training in the UK has undergone major 
changes recently, with the introduction of work- 
based assessments (WBAs) being one of the most sig- 
nificant changes. 1 Since their introduction, the use of 
the WBAs has become more widespread. WBAs, 
which were initially trialled with foundation doctors, 
are now a routine part of training for medical trainees 
of all grades and all specialities in the UK. 

The popularity of WBAs stems from the fact that 
they are tests of the highest level of clinical compe- 
tence. Miller 2 proposed a pyramid for assessing clin- 
ical competence. The successive levels of this pyramid 
from the lowest level are knowledge (knows), compe- 
tence (knows how), performance (shows how) and 
action (does). WBAs are tests of the highest level of 
clinical competence. For example, the multi-source 
feedback (MSF) assessment tests the performance 
of doctors in their normal practice, i.e. what a 
doctor 'does'. Another reason for the popularity of 
WBAs is that they also assess non-technical skills 
such as professionalism, decision making and time 
keeping. 3 

As the use of WBAs is expanding, the practical 
difficulties with implementing them have begun to 
emerge. A major problem is the failure of trainees, 



trainers and training programmes to adequately 
engage with them. The aim of this review was to iden- 
tify (1) the reasons for the poor engagement with 
WBAs and (2) how these can be overcome (Table 1). 

Methods 

A pragmatic review of WBAs was undertaken. The 
review was limited to WBAs that are routinely used in 
medicine: Direct Observation of Procedural Skills 
(DOPs), Mini-Clinical Evaluation eXercises (mini- 
CEXs), Case-Based Discussions (CBDs) and MSFs. 
The relevant literature pertaining to WBAs in the 
medical setting was identified using the online data- 
bases: MEDLINE®, Google Scholar and Web of 
Knowledge. 

Time 

WBAs are time consuming. A study on three com- 
monly used WBAs discovered that the mean total 
time required for the completion of a mini-CEX, 
MSF and DOP, respectively, were 25min, 6.8 min 
and the duration of the procedure plus a further 
third of this time for feedback. 4 These are likely to 
be underestimates of the true time required for the 
completion of WBAs, as they only took the time 
required for the assessment and feedback into consid- 
eration and not the time required for locating the 
assessor or identifying suitable cases. As WBAs are 
time consuming and there is no dedicated time in 
clinical practice when they can be undertaken, it is 
unsurprising then that the lack of time is a major 
reason that prevents the completion of WBAs. 4 ' 5 

A potential solution is to have 'time written into 
the job plans of clinical teachers'. 5 Another solution 
for the lack of assessors' time is to encourage trainees 
'to use as many different assessors as possible, as 
this . . . spreads the workload'. 6 Training assessors in 
WBAs is required to increase the pool of the asses- 
sors, as a lack of training in WBAs was one of the 
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Table 1 . How can engagement with work-based assessments be 


improved. 


Problem 


Solution 


Lack of time 


Time for WBAs needs to written into the job plans of asses- 
sors and trainees. 
Use multiple assessors as this spreads the workload. 


WBAs place an unnecessary administrative burden 
They are not easy to use 

Lack of access to a computer 

Lack of sufficient training in the assessment methodology 


WBAs need to be simplified so as to make them more user- 
friendly. 

Use of smartphones would ensure that WBAs are completed 
even when there is no access to computers. 

Assessor needs to be trained in WBAs with the most favoured 
form of training being workshops/seminars and practical 


Trainees have a pessimistic view of the education value of 
WBAs 

Lock of effective feedback 

Inappropriate use of WBAs in summative assessments 

Perceived lack of validity, reliability and evidence for these 
instruments 


Assessors need to be educated about the importance of 
feedback and be taught the skills required to provide high- 
quality feedback. 

A cultural shift is needed such that WBAs are used solely as 
formative assessments, with emphasis on feedback. 

Both trainee and trainer need to be educated about the evi- 
dence base for these instruments. 


Lack of enthusiasm from assessors 


Assessors with low enthusiasm should be identified and 

reasons for their low enthusiasm should be addressed. 
Assessors with high enthusiasm should be rewarded. 



common reasons cited by assessors who refused to 
undertake WBAs. 7 To further increase the pool of 
assessors, non-medical assessors may be used. 
Unfortunately, early experience of using non-medical 
assessors was that they are even less familiar with 
WBAs than their medical counterparts. Only 6% of 
non-medical assessors were rated by trainees as 
having sufficient knowledge of WBAs. 1 If the practice 
of using non-medical assessors is to be continued, 
then targeted training of the non-medical assessors 
is vital. Such a move is likely to receive the support 
of non-medical assessors because in a recent survey of 
non-medical assessors, 92% felt that training would 
be a help. 8 

In addition to addressing the lack of assessors' 
time, the lack of time for trainees to undertake 
WBAs also needs to be addressed. In a survey of 
psychiatric trainees, 70% of respondents reported 
that WBAs had a negative impact on the time avail- 
able for their clinical duties. 1 This finding is surpris- 
ing because in theory WBAs are observations of 
routine clinical practice and not an extra event. 
Therefore, careful planning is required so that this 
training need, WBAs, is met without compromising 
clinical care. The most favoured solution by trainees 
was to have regular protected time with their assessor 



that is free of clinical commitments, when they could 
undertake WBAs. 1 ' 9 Including time for WBAs in job 
plans of assessors and trainees will be costly, 'which 
in the current financial climate may be difficult to 
achieve 1 . 10 Bindal et al. 7 suggest pre-planning of 
WBAs into clinical practice as a way of creating 
time for WBAs but without the financial cost asso- 
ciated with including time in assessors' and trainees' 
job plans for WBAs. 

A radical solution to the lack of time problem is to 
reduce the number of each type of WBAs that a trai- 
nee needs to undertake. Such a strategy should be 
pursued with caution. Owing to inter-assessor vari- 
ations in marking, WBA 'needs to be repeated on 
several occasions for it to be a reliable measure'. 3 It 
then seems logical that training assessors in using 
WBAs will improve marking consistency and its edu- 
cational value, which will then make it possible to 
reduce the number of each type of WBAs that a trai- 
nee needs to undertake. The effect of assessor training 
on performance was assessed by Cook et al., who 
evaluated the effect of an assessor training workshop 
on inter-assessor reliability and accuracy of mini- 
CEX scores. Surprisingly, they discovered there was 
no difference in inter-assessor reliability of mini-CEX 
scores between the assessors who had attended 
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workshop and the control group. Possible explan- 
ations for this observation offered by Cook et al. 11 
include a small size (31), which means that the study 
lacked the power to detect small changes, and an 
ineffective workshop format. A more worrying 
explanation is that 'physician raters are impervious 
to training'. 12 A great deal of effort is being directed 
to determine the barriers to training assessors and 
how these can be overcome, with the ultimate aim 
of reducing the minimum number of WBAs that 
needs to be undertaken. Such an approach is funda- 
mentally flawed. It fails to recognize these multiple 
encounters with assessors where different issues are 
discussed as beneficial. These multiple assessments 
act as multiple sources of feedback for the trainees, 
allowing them to truly identify their strengths, the 
areas they need to work on and how they should 
make the changes. Therefore, the key is not to 
reduce the minimum number of WBAs but to make 
them easier to use. 

Administrative burden 

Another possible reason for the failure of trainees to 
engage with WBAs is the bureaucratic burden of the 
assessments. Trainees' experience of the foundation 
programme learning portfolio is that it involves 
excessive paperwork, a view strongly agreed by 
84% and 77% of trainees from the 2004 and the 
2005 cohort, respectively. 13 While WBAs are an 
important part of the foundation programme learn- 
ing portfolio, there are also several other compo- 
nents. Therefore, from the work by Hrisos et al., 13 
it is not possible to make conclusive statements about 
the bureaucratic burden of WBAs. However, in a 
survey of medical specialist registrars (SpRs), a 
number of trainees reported that WBAs created an 
excessive administrative workload. 4 

The fact that WBAs are not easy to use 1 and are 
not user-friendly 14 is felt by experts to be the main 
reason for the excessive administrative workload. 
WBAs need to be simplified 'so as to make them 
more user-friendly in busy clinical settings where 
patient care is the first priority and trainee assessment 
of less importance'. 15 A fear with simplification of 
WBAs is that this may dilute their educational 
value. However, Driessen et al. 16 developed and 
implemented a lean non-bureaucratic assessment pro- 
gramme that had a high educational value, was 
robust and was well liked by trainees. Hence, we 
urge that the current medical assessment programmes 
are critically reviewed and modified so that they 
incorporate Driessen's principles. 

Another contributory factor to the administrative 
burden of the WBAs is the lack of access to 



computers. Only 11% of respondents to a survey of 
psychiatric trainees' views of WBA agreed with the 
statement that they had easy access to a computer. 1 A 
solution to the lack of easy access to computers is to 
embrace the use of the smartphone. Smartphone sales 
have grown rapidly over the last five years, so much 
so that in 2011, more smartphones were sold than 
personal computers. 17 Smartphones can be used to 
complete WBAs even when there is no access to a 
computer, and this should increase the completion 
rates of WBAs. 18 

A further reason for the administrative burden of 
WBAs is the lack of sufficient training in the assess- 
ment methodology. While it is tempting to speculate 
that training the trainees will increase participation, 
Pelgrim et al. 19 acknowledge the absence of any evi- 
dence to support this claim. Assessors also need to be 
trained in WBAs to further reduce the administrative 
burden. The need for assessor training in WBAs is 
supported by the fact that 22% of assessors who 
responded to a survey on WBAs reported that they 
had received no training and yet they routinely under- 
take WBAs. 20 Of the assessors who had training, only 
55% felt confident or very confident in undertaking 
WBAs, 20 which suggests the need for further training 
in WBAs. The form of training most favoured by 
assessors for this purpose was workshops/seminars 
and practical hands-on marking. 20 The popularity 
of workshops stems from the fact that they are flex- 
ible, promote active learning and a variety of teaching 
methods are employed within this format, including 
interactive lectures, small-group discussions and 
exercises, role-plays and simulations. 21 

Educational value 

A recurring theme is that trainees are unconvinced 
of the education value of WBAs. A survey of 
Foundation Year 2 trainees (FY2) demonstrated 
that 60.9%, 56.1% and 39.1% of trainees felt that 
DOPs, mini-CEXs and CBDs, respectively, were use- 
less or very useless. 22 A similar result was observed on 
a survey of GP specialist trainees (GPSTs). Only 55% 
of GPSTs viewed WBAs as a useful educational tool. 23 
In fact, some trainees view WBAs as a hindrance. In a 
survey of British trainee surgeons, 63.6% of respond- 
ents reported that the completion of WBAs had an 
adverse impact on their training opportunities. 14 

One explanation for trainees' pessimistic view of 
the education value of WBAs is the lack of effective 
feedback. For the majority of trainees (94% for 
verbal and 74% for written), feedback is a useful 
element of WBAs but a number of assessors (27.5% 
for verbal and 61% for written) are unable to give 
effective feedback. 23 Trainees' perception that 
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assessors are unable to give effective feedback 
appears to be an accurate reflection of reality. Ali 
et al. 24 retrospectively analysed the feedback that 
had been recorded in the WBAs undertaken by urol- 
ogy trainees and discovered that there was a lack of 
feedback and when there was feedback it was of poor 
quality. Holmboe et al. 25 evaluated the feedback that 
was given after a mini-CEX and found it to be poor. 
Only in 34% of encounters did the assessor ask for 
self-assessment from the trainee and only in 8% of 
encounters was an action plan formulated. 

Both a lack of appreciation by assessors of the 
importance of feedback and the fact that they may 
not be skilled in the process of providing high-quality 
feedback may explain why assessors fail to give effect- 
ive feedback. 15 However, there is conflicting evidence 
in the literature as to the impact of assessor training 
on the quality of feedback. Salerno et al. 26 discovered 
that the rate of specific feedback increased from 22% 
to 38% after faculty development seminars. In con- 
trast, Holmboe et al. 27 discovered that the rate of 
specific feedback remained unchanged after interven- 
tion. A plausible explanation for this observation is 
that intensive interventions are required to change 
assessors' behaviour and improve the quality of 
their feedback. Holmboe et al.'s intervention 27 was 
a 20-min didactic teaching session on feedback, 
while Salerno et al.'s intervention 26 consisted of 
three 90-min interactive seminars scheduled one 
week apart. Therefore, educating assessors about 
feedback principles and techniques needs to be inten- 
sive if it is to improve the quality of their feedback 
and to increase the educational value of WBAs. 

The seniority of the assessor influences the quality of 
feedback. According to trainees, the more senior the 
assessors, the more useful was feedback that they 
received. 9 However, FY2 trainees found that the 
most senior of these assessors (consultants) were reluc- 
tant to undertake WBAs. Only 13% of respondents in 
a survey of a FY2 trainees reported that their consult- 
ants were keen to complete WBAs. 9 Apathy may be a 
factor in poor consultant contribution. However, work 
by Finall 28 suggests that this may not be the case, as she 
demonstrated that consultants valued WBAs. 
However, the conclusion drawn from Finall's work 
should be tempered with the knowledge that her 
survey only captured the view of seven consultant 
histopathologists from a single hospital. Work by 
Menon et al. potentially offers an explanation for 
the poor consultant contribution reported by FY2 
trainees. In his survey of STs, 91% rated consultant 
assessors as willing to complete WBAs. 1 This 
finding suggests that poor consultant contribution 
to WBAs may be limited to more junior trainees. 
A possible explanation for this is that there is less 



interaction between consultants and junior trainees 
as compared to senior trainees owing to the different 
working relationships of these grades with their con- 
sultants and the shorter placements of junior trainees. 
Therefore, consultants should specially be encouraged 
to contribute more to WBAs of their more junior 
trainees. 

Further reasons for trainees' pessimistic view of the 
education value of WBAs are a perceived lack of val- 
idity, reliability and evidence base for these instru- 
ments, 1 and this may account for failure to engage 
with them. To address this, (1) threats to the validity 
and reliability in the workplace should be identified 
and addressed and (2) trainees need to be educated 
about the evidence base for these instruments. 

Schuwirth and van der Vleuten 29 highlighted a 
more generic reason for the limited educational 
value of WBAs. They argue that in medicine 'assess- 
ment is synonymous with punitive examinations 
whose sole purpose is to pass or fail candidates' 
and this 'has made us fear assessment'. 29 For WBA 
to be effective, it must be conducted as formative 
assessments with emphasis on trainees to engage 
with the feedback to improve themselves. This is unli- 
kely to take place in a culture where assessments are 
feared. At present, trainees are likely to fear WBAs as 
they contribute to their annual review of compe- 
tences, i.e. WBAs are being used in summative assess- 
ments. Hence, a cultural shift is required in medicine 
such that WBAs are viewed in positive light and used 
solely as formative assessments to improve perform- 
ance. One way to ensure that WBAs are solely used as 
formative assessment is to remove all links to ratings 
from them. In the absence of ratings, it is impossible 
to use WBAs in summative assessments, and this 
would encourage a more honest engagement with 
WBAs. In fact, this has already happened with the 
foundation programme WBAs, where ratings have 
been removed. They are now used solely as super- 
vised learning events with candidate feedback. 

Assessors' enthusiasm 

Assessors' enthusiasm for WBAs is likely to influence 
trainees' engagement. Nisar and Scott. 30 demon- 
strated that core trainees valued trainers' enthusiasm 
as a key trainer attribute. Trainees have reported a 
lack of enthusiasm from trainers ' for WBAs. 1 Some 
assessors view WBAs as tick-box exercises while 
others have genuine passion for these assessments. A 
two-pronged strategy is required. Assessors who have 
low enthusiasm for WBAs should be identified and 
reasons for their low enthusiasm should be explored. 
Trainers with high enthusiasm for WBAs should be 
rewarded so that their enthusiasm is maintained. 
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Discussion 

This review has its limitations. The intention of this 
review was that it was a pragmatic review of litera- 
ture, rather than a systemic review of the literature. 
Hence, there is a possibility that not all relevant 
studies have been identified in spite of searching 
multiple databases and manually searching the ref- 
erence lists. Furthermore, the quality of this review 
is limited by the quality of the studies. The majority 
of the studies reviewed were cross-sectional surveys 
of trainees' and assessors' view of WBAs. A concern 
with the conclusions drawn from such studies is 
whether the views expressed in the surveys are 
truly representative of the views of the sample popu- 
lation. To overcome this, a number of surveys have 
used free text spaces, which allows respondents to 
express their view over and above which that was 
specifically asked. It is likely that as the surveys were 
anonymous the participants expressed their views 
freely. Furthermore, the unusually high response 
rates for these surveys suggest that the views 
expressed by the participants of the surveys are 
likely to be representative of the sample population. 
A further concern with drawing conclusions from 
trainees' view of WBAs is that their perception of 
the issues may be different from the reality. 
However, there is some evidence to suggest that 
the trainees' view of WBAs is an accurate reflection 
of reality. Work by Wilkinson et al. 4 has shown 
WBAs to be time consuming and this supports the 
claim that lack of time is a major reason that pre- 
vents the completion of WBAs. Similarly, work by 
Holmboe et al. 25 offers support to trainees' claim 
that the quality of the feedback they receive is 
poor. Even if trainees' perceptions of WBAs are dif- 
ferent from the reality it is nevertheless important to 
be aware of them. Unless these misconceptions are 
corrected, trainees are unlikely to engage effectively 
with WBAs. 

At present, there are a number of issues that pre- 
vent trainees, trainers and training programmes from 
adequately engaging with WBAs. Failure to address 
these issues will limit the long-term value, usefulness 
and success of WBAs. The importance and relevance 
of this review is that it highlights these barriers as well 
as the strategies that can be employed to overcome 
them. A range of strategies have been considered 
along with their merits and pitfalls, which allows indi- 
viduals and institutions to select the strategies that 
suit their specific needs. 

Conclusion 

The application of WBAs in medicine has seen a 
rapid growth in the last few years. However, trainees, 



trainers and training programmes are not engaging 
wholeheartedly with them, which limits the educa- 
tional value of these instruments. Fortunately, the 
reasons for this are beginning to emerge, along with 
potential solutions. 
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